The technology described herein relates to data processing systems and in particular to display controllers for data processing systems.
FIG. 1 shows an exemplary data processing system that comprises a central processing unit (CPU) 7, a graphics processing unit (GPU) 2, a video codec 1, a display controller 5, and a memory controller 8. As shown in FIG. 1, these units communicate via an interconnect 9 and have access to off-chip memory 3. In use of this system the GPU 2, video codec 1 and/or CPU 7 will generate surfaces (images) to be displayed and store them, via the memory controller 8, in respective frame buffers in the off-chip memory 3. The display controller 5 will then read those surfaces as input layers from the frame buffers in the off-chip memory 3 via the memory controller 8, process the input surfaces appropriately and send them to a display 4 for display.
FIG. 2 shows an exemplary data path for the processing of the input surfaces for display in the display controller 5. It is assumed in this example that the display controller 5 can take as inputs for a given output surface to be displayed a plurality of input surfaces (layers), and includes, inter alia, a composition engine (stage) 22 that is able to compose one or more input surfaces (layers) (e.g. generated by the GPU 2 and/or video codec 1) to provide a composited output frame for display.
As shown in FIG. 2, the display controller includes a DMA (Direct Memory Access) read unit 20 that reads data of input surfaces to be displayed and provides it appropriately to respective sets of latency “hiding” FIFOs 21. (The latency hiding FIFOs 21 provide “latency” buffering in the display processing path to allow for potential latency in retrieving the required input surface data from memory. There is one set of latency FIFOs 21 for each “layer” that the display controller can take as an input for its processing.)
The input surfaces that the display controller 5 processes to provide the output surface for display will be generated, as discussed above, e.g. by the video codec 1, CPU 7 and/or GPU 2 of the overall data processing system, and stored as respective frame buffers in the main memory 3 of the data processing system.
Each input surface will occupy a certain amount of physical memory of the main memory 3 and will need to be read from that physical memory when it is to be processed by the display controller 5.
In order to access data stored in memory, many data processing systems use so-called “virtual” memory addressing arrangements, in which the address used in a given memory access request is translated from a virtual memory address used by the memory access initiator (the unit requesting the memory access) to a corresponding physical memory address used by the memory system. To perform the translation between the (virtual) address used for a memory access request and the corresponding physical memory address where the data is actually stored, a set of address translation data that maps virtual addresses used in memory accesses to corresponding physical memory addresses is usually stored.
This process is typically performed in a so-called memory management unit of the data processing system. The memory management unit operates to allocate physical memory for data storage in given units of memory size, typically referred to as “memory pages” or “pages”, and associates corresponding virtual memory addresses (pages) with the physical memory addresses (pages) where the data is actually stored. Accordingly, the address translation data which maps virtual addresses used in memory accesses to corresponding physical memory addresses typically maps virtual memory page addresses to corresponding physical memory page addresses. Thus the memory address translation data is typically in the form of so-called page tables. There can be multiple levels of page tables (which, with the exception of the base page table, are typically also stored in virtual memory).
The memory management unit pages are typically allocated at a predefined granularity (size) in order to allow for memory allocations of different sizes to be created and released by the memory management unit. Each input surface will, typically, occupy plural memory pages. The memory page allocation for an input surface will frequently correspond to physical pages occupying non-contiguous address ranges in memory.
The full set of memory management address translation data (the full MMU page table) is typically stored in main memory, and indicates which physical memory address range corresponds to a given virtual memory address range that may be used by a memory access initiator.
However, to facilitate more rapid accessing of the memory access address translation data, memory systems and memory management units typically include a cache memory, often referred to as a translation lookaside buffer (TLB), in which a small amount of address translation data (page table entries) is stored, so as to allow faster access to that address translation data and to thereby speed up the address translation and checking process. (If the required address translation data is not stored in the cache (TLB), then that data must be fetched from the main address translation data (page tables) in main memory, which can take a more significant period of time.)
When a frame is to be displayed, the input surfaces that form the input layers are composed in the display composition stage 22 to provide a composited output surface for display. The composited output surface (i.e. the frame that is to be displayed) is then subject to display timing control 23 (e.g. the inclusion of appropriate horizontal and vertical blanking periods), and then provided to the display output interface of the display controller 5 for provision to the display 4 for display.
This process is repeated for each frame that needs to be displayed, e.g. at a rate of 30 or 60 frames per second.
As such display processing is a real-time operation, the display controller 5 needs to deliver the pixel data to be displayed to the display 4 (to the display output) regularly, in each clock cycle triggering the display output from the display controller. If valid pixel data is not available when the display output is triggered, then so-called “under-run” occurs, i.e. there is insufficient data for sending to the display. In this case, some default, “error” operation is usually performed, such as simply displaying a default arbitrary colour for those pixels for which “real” pixel data is not available.
Such “under-run” can occur, for example, because of latencies in fetching the input surface data from memory, such that the required data has not been fetched and/or has not completed its processing, by the time it is required to be displayed.
The Applicants believe that there remains scope for improvements to the operation of display controllers when providing frames for display.
Like reference numerals are used for like components throughout the drawings, where appropriate.